Skip to content

Add check_debugger job to CI #8240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
smola merged 1 commit into from
Jan 20, 2025
Merged

Add check_debugger job to CI #8240

smola merged 1 commit into from
Jan 20, 2025

Conversation

@smola
Copy link
Member

@smola smola commented Jan 17, 2025

What Does This Do

Adds a check_debugger job to CI.

Add forbidden apis suppressions required to pass.

Motivation

It seems that ever since we split the check job, the debugger variant was never added.

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Jan 17, 2025
@smola smola force-pushed the smola/check-debugger branch from 210063b to 2b19a80 Compare January 17, 2025 11:31
@pr-commenter
Copy link

pr-commenter bot commented Jan 17, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1737359609 1737359981
end_time 2025-01-20T07:54:45 2025-01-20T08:00:58
git_branch master smola/check-debugger
git_commit_sha a0e340b 8718354
start_time 2025-01-20T07:53:30 2025-01-20T07:59:42
See matching parameters
Baseline Candidate
ci_job_id 772237340 772237340
ci_pipeline_id 53446944 53446944
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1737358992 1737358992

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 1 metrics, 9 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+160.949µs; +393.807µs] or [+1.607%; +3.931%]		 worse
[+173.364µs; +412.862µs] or [+1.704%; +4.059%]		 worse
[+154.882µs; +403.541µs] or [+1.507%; +3.926%]		 worse
[+112.390µs; +517.796µs] or [+1.059%; +4.878%]		 worse
[-4.306op/s; -1.199op/s] or [-4.435%; -1.235%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-16.901µs; +40.929µs] or [-6.424%; +15.558%]		 unstable
[-25.397µs; +55.935µs] or [-8.436%; +18.580%]		 unstable
[-33.978µs; +65.978µs] or [-10.807%; +20.984%]		 unstable
[-95.121µs; +118.840µs] or [-16.217%; +20.261%]		 same
scenario:basic unstable
[-40.022µs; +9.338µs] or [-13.708%; +3.198%]		 unstable
[-46.602µs; +14.551µs] or [-14.381%; +4.490%]		 unstable
[-61.091µs; +20.319µs] or [-18.021%; +5.994%]		 unstable
[-1096.757µs; +360.333µs] or [-121.746%; +39.999%]		 unstable
[-207.061op/s; +207.061op/s] or [-8.282%; +8.282%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (301.055 µs) : 281, 321
.   : milestone, 301,
basic (324.048 µs) : 284, 364
.   : milestone, 324,
loop (10.172 ms) : 10057, 10286
.   : milestone, 10172,
section candidate
noprobe (316.324 µs) : 265, 367
.   : milestone, 316,
basic (308.022 µs) : 296, 321
.   : milestone, 308,
loop (10.465 ms) : 10357, 10572
.   : milestone, 10465,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 301.055 µs [281.234 µs, 320.875 µs]
basic 324.048 µs [283.692 µs, 364.403 µs]
loop 10.172 ms [10.057 ms, 10.286 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 316.324 µs [265.211 µs, 367.436 µs]
basic 308.022 µs [295.529 µs, 320.515 µs]
loop 10.465 ms [10.357 ms, 10.572 ms]

@pr-commenter
Copy link

pr-commenter bot commented Jan 17, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/check-debugger
git_commit_date 1737139405 1737358992
git_commit_sha 0b239ae 8718354
release_version 1.46.0-SNAPSHOT~0b239ae5cc 1.46.0-SNAPSHOT~8718354560
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737361352 1737361352
ci_job_id 772237334 772237334
ci_pipeline_id 53446944 53446944
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 3 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~8718354560, baseline=1.46.0-SNAPSHOT~0b239ae5cc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1064072
Total [baseline] (10.514 s) : 0, 10514048
Agent [candidate] (1.059 s) : 0, 1058907
Total [candidate] (10.455 s) : 0, 10454591
section appsec
Agent [baseline] (1.191 s) : 0, 1191337
Total [baseline] (10.727 s) : 0, 10726893
Agent [candidate] (1.189 s) : 0, 1188816
Total [candidate] (10.735 s) : 0, 10734885
section iast
Agent [baseline] (1.193 s) : 0, 1193352
Total [baseline] (11.009 s) : 0, 11009369
Agent [candidate] (1.185 s) : 0, 1184500
Total [candidate] (10.968 s) : 0, 10967534
section profiling
Agent [baseline] (1.261 s) : 0, 1260593
Total [baseline] (10.88 s) : 0, 10879847
Agent [candidate] (1.256 s) : 0, 1255911
Total [candidate] (10.869 s) : 0, 10868999
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent appsec 1.191 s 127.265 ms (12.0%)
Agent iast 1.193 s 129.279 ms (12.1%)
Agent profiling 1.261 s 196.521 ms (18.5%)
Total tracing 10.514 s -
Total appsec 10.727 s 212.845 ms (2.0%)
Total iast 11.009 s 495.321 ms (4.7%)
Total profiling 10.88 s 365.799 ms (3.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent appsec 1.189 s 129.908 ms (12.3%)
Agent iast 1.185 s 125.593 ms (11.9%)
Agent profiling 1.256 s 197.004 ms (18.6%)
Total tracing 10.455 s -
Total appsec 10.735 s 280.294 ms (2.7%)
Total iast 10.968 s 512.943 ms (4.9%)
Total profiling 10.869 s 414.408 ms (4.0%) 
gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~8718354560, baseline=1.46.0-SNAPSHOT~0b239ae5cc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (719.695 ms) : 0, 719695
BytebuddyAgent [candidate] (715.824 ms) : 0, 715824
GlobalTracer [baseline] (257.325 ms) : 0, 257325
GlobalTracer [candidate] (255.878 ms) : 0, 255878
AppSec [baseline] (56.474 ms) : 0, 56474
AppSec [candidate] (55.629 ms) : 0, 55629
Remote Config [baseline] (745.942 µs) : 0, 746
Remote Config [candidate] (733.589 µs) : 0, 734
Telemetry [baseline] (14.872 ms) : 0, 14872
Telemetry [candidate] (15.791 ms) : 0, 15791
section appsec
BytebuddyAgent [baseline] (732.873 ms) : 0, 732873
BytebuddyAgent [candidate] (731.342 ms) : 0, 731342
GlobalTracer [baseline] (253.583 ms) : 0, 253583
GlobalTracer [candidate] (252.955 ms) : 0, 252955
AppSec [baseline] (171.318 ms) : 0, 171318
AppSec [candidate] (170.949 ms) : 0, 170949
Remote Config [baseline] (660.856 µs) : 0, 661
Remote Config [candidate] (679.687 µs) : 0, 680
Telemetry [baseline] (8.175 ms) : 0, 8175
Telemetry [candidate] (8.183 ms) : 0, 8183
IAST [baseline] (19.553 ms) : 0, 19553
IAST [candidate] (19.38 ms) : 0, 19380
section iast
BytebuddyAgent [baseline] (839.388 ms) : 0, 839388
BytebuddyAgent [candidate] (833.128 ms) : 0, 833128
GlobalTracer [baseline] (248.783 ms) : 0, 248783
GlobalTracer [candidate] (247.166 ms) : 0, 247166
AppSec [baseline] (58.562 ms) : 0, 58562
AppSec [candidate] (58.185 ms) : 0, 58185
Remote Config [baseline] (674.444 µs) : 0, 674
Remote Config [candidate] (674.044 µs) : 0, 674
Telemetry [baseline] (8.998 ms) : 0, 8998
Telemetry [candidate] (8.788 ms) : 0, 8788
IAST [baseline] (21.849 ms) : 0, 21849
IAST [candidate] (21.482 ms) : 0, 21482
section profiling
BytebuddyAgent [baseline] (706.662 ms) : 0, 706662
BytebuddyAgent [candidate] (705.178 ms) : 0, 705178
GlobalTracer [baseline] (350.94 ms) : 0, 350940
GlobalTracer [candidate] (349.669 ms) : 0, 349669
AppSec [baseline] (54.969 ms) : 0, 54969
AppSec [candidate] (54.247 ms) : 0, 54247
Remote Config [baseline] (700.072 µs) : 0, 700
Remote Config [candidate] (726.758 µs) : 0, 727
Telemetry [baseline] (8.874 ms) : 0, 8874
Telemetry [candidate] (8.782 ms) : 0, 8782
ProfilingAgent [baseline] (96.383 ms) : 0, 96383
ProfilingAgent [candidate] (95.291 ms) : 0, 95291
Profiling [baseline] (96.409 ms) : 0, 96409
Profiling [candidate] (95.315 ms) : 0, 95315
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~8718354560, baseline=1.46.0-SNAPSHOT~0b239ae5cc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1057878
Total [baseline] (8.624 s) : 0, 8624432
Agent [candidate] (1.055 s) : 0, 1054723
Total [candidate] (8.611 s) : 0, 8611016
section iast
Agent [baseline] (1.182 s) : 0, 1181599
Total [baseline] (9.187 s) : 0, 9187364
Agent [candidate] (1.182 s) : 0, 1181901
Total [candidate] (9.225 s) : 0, 9224693
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.194 s) : 0, 1194142
Total [baseline] (9.187 s) : 0, 9187481
Agent [candidate] (1.185 s) : 0, 1184932
Total [candidate] (9.184 s) : 0, 9183825
section iast_TELEMETRY_OFF
Agent [baseline] (1.18 s) : 0, 1179716
Total [baseline] (9.203 s) : 0, 9202605
Agent [candidate] (1.182 s) : 0, 1181614
Total [candidate] (9.219 s) : 0, 9218657
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.058 s -
Agent iast 1.182 s 123.721 ms (11.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.194 s 136.264 ms (12.9%)
Agent iast_TELEMETRY_OFF 1.18 s 121.838 ms (11.5%)
Total tracing 8.624 s -
Total iast 9.187 s 562.931 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.187 s 563.049 ms (6.5%)
Total iast_TELEMETRY_OFF 9.203 s 578.172 ms (6.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent iast 1.182 s 127.177 ms (12.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.185 s 130.209 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.182 s 126.891 ms (12.0%)
Total tracing 8.611 s -
Total iast 9.225 s 613.677 ms (7.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.184 s 572.809 ms (6.7%)
Total iast_TELEMETRY_OFF 9.219 s 607.641 ms (7.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~8718354560, baseline=1.46.0-SNAPSHOT~0b239ae5cc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (715.326 ms) : 0, 715326
BytebuddyAgent [candidate] (714.059 ms) : 0, 714059
GlobalTracer [baseline] (255.96 ms) : 0, 255960
GlobalTracer [candidate] (255.873 ms) : 0, 255873
AppSec [baseline] (56.585 ms) : 0, 56585
AppSec [candidate] (56.302 ms) : 0, 56302
Remote Config [baseline] (727.898 µs) : 0, 728
Remote Config [candidate] (750.453 µs) : 0, 750
Telemetry [baseline] (14.334 ms) : 0, 14334
Telemetry [candidate] (12.718 ms) : 0, 12718
section iast
BytebuddyAgent [baseline] (830.914 ms) : 0, 830914
BytebuddyAgent [candidate] (831.378 ms) : 0, 831378
GlobalTracer [baseline] (246.48 ms) : 0, 246480
GlobalTracer [candidate] (246.557 ms) : 0, 246557
AppSec [baseline] (58.17 ms) : 0, 58170
AppSec [candidate] (58.2 ms) : 0, 58200
Remote Config [baseline] (658.36 µs) : 0, 658
Remote Config [candidate] (654.423 µs) : 0, 654
Telemetry [baseline] (8.832 ms) : 0, 8832
Telemetry [candidate] (8.759 ms) : 0, 8759
IAST [baseline] (21.564 ms) : 0, 21564
IAST [candidate] (21.378 ms) : 0, 21378
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (840.08 ms) : 0, 840080
BytebuddyAgent [candidate] (833.213 ms) : 0, 833213
GlobalTracer [baseline] (248.211 ms) : 0, 248211
GlobalTracer [candidate] (247.625 ms) : 0, 247625
AppSec [baseline] (59.043 ms) : 0, 59043
AppSec [candidate] (58.105 ms) : 0, 58105
Remote Config [baseline] (677.861 µs) : 0, 678
Remote Config [candidate] (669.38 µs) : 0, 669
Telemetry [baseline] (8.993 ms) : 0, 8993
Telemetry [candidate] (8.827 ms) : 0, 8827
IAST [baseline] (22.021 ms) : 0, 22021
IAST [candidate] (21.464 ms) : 0, 21464
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (829.77 ms) : 0, 829770
BytebuddyAgent [candidate] (830.51 ms) : 0, 830510
GlobalTracer [baseline] (246.955 ms) : 0, 246955
GlobalTracer [candidate] (247.317 ms) : 0, 247317
AppSec [baseline] (57.838 ms) : 0, 57838
AppSec [candidate] (58.351 ms) : 0, 58351
Remote Config [baseline] (660.809 µs) : 0, 661
Remote Config [candidate] (677.43 µs) : 0, 677
Telemetry [baseline] (8.683 ms) : 0, 8683
Telemetry [candidate] (8.682 ms) : 0, 8682
IAST [baseline] (20.851 ms) : 0, 20851
IAST [candidate] (21.059 ms) : 0, 21059
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-01-20T07:52:29 2025-01-20T07:59:33
git_branch master smola/check-debugger
git_commit_date 1737139405 1737358992
git_commit_sha 0b239ae 8718354
release_version 1.46.0-SNAPSHOT~0b239ae5cc 1.46.0-SNAPSHOT~8718354560
start_time 2025-01-20T07:52:15 2025-01-20T07:59:19
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737360331 1737360331
ci_job_id 772237335 772237335
ci_pipeline_id 53446944 53446944
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~8718354560, baseline=1.46.0-SNAPSHOT~0b239ae5cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (378.656 µs) : 358, 399
.   : milestone, 379,
iast (505.304 µs) : 484, 527
.   : milestone, 505,
iast_FULL (745.975 µs) : 724, 768
.   : milestone, 746,
iast_GLOBAL (555.192 µs) : 533, 578
.   : milestone, 555,
iast_HARDCODED_SECRET_DISABLED (505.725 µs) : 484, 527
.   : milestone, 506,
iast_INACTIVE (464.04 µs) : 442, 486
.   : milestone, 464,
iast_TELEMETRY_OFF (491.321 µs) : 470, 513
.   : milestone, 491,
tracing (446.864 µs) : 426, 468
.   : milestone, 447,
section candidate
no_agent (374.04 µs) : 353, 395
.   : milestone, 374,
iast (506.957 µs) : 485, 529
.   : milestone, 507,
iast_FULL (742.111 µs) : 720, 764
.   : milestone, 742,
iast_GLOBAL (556.209 µs) : 534, 578
.   : milestone, 556,
iast_HARDCODED_SECRET_DISABLED (512.401 µs) : 491, 534
.   : milestone, 512,
iast_INACTIVE (463.48 µs) : 442, 485
.   : milestone, 463,
iast_TELEMETRY_OFF (498.227 µs) : 476, 520
.   : milestone, 498,
tracing (451.528 µs) : 430, 473
.   : milestone, 452,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.656 µs [358.359 µs, 398.953 µs] -
iast 505.304 µs [483.508 µs, 527.1 µs] 126.648 µs (33.4%)
iast_FULL 745.975 µs [723.774 µs, 768.177 µs] 367.319 µs (97.0%)
iast_GLOBAL 555.192 µs [532.742 µs, 577.642 µs] 176.536 µs (46.6%)
iast_HARDCODED_SECRET_DISABLED 505.725 µs [484.198 µs, 527.251 µs] 127.069 µs (33.6%)
iast_INACTIVE 464.04 µs [441.742 µs, 486.338 µs] 85.384 µs (22.5%)
iast_TELEMETRY_OFF 491.321 µs [469.722 µs, 512.919 µs] 112.665 µs (29.8%)
tracing 446.864 µs [426.12 µs, 467.608 µs] 68.208 µs (18.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 374.04 µs [353.377 µs, 394.702 µs] -
iast 506.957 µs [485.031 µs, 528.882 µs] 132.917 µs (35.5%)
iast_FULL 742.111 µs [720.042 µs, 764.181 µs] 368.072 µs (98.4%)
iast_GLOBAL 556.209 µs [534.092 µs, 578.326 µs] 182.169 µs (48.7%)
iast_HARDCODED_SECRET_DISABLED 512.401 µs [490.653 µs, 534.149 µs] 138.362 µs (37.0%)
iast_INACTIVE 463.48 µs [441.692 µs, 485.268 µs] 89.44 µs (23.9%)
iast_TELEMETRY_OFF 498.227 µs [476.447 µs, 520.007 µs] 124.187 µs (33.2%)
tracing 451.528 µs [430.452 µs, 472.603 µs] 77.488 µs (20.7%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~8718354560, baseline=1.46.0-SNAPSHOT~0b239ae5cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.369 ms) : 1349, 1389
.   : milestone, 1369,
appsec (1.748 ms) : 1724, 1772
.   : milestone, 1748,
appsec_no_iast (1.755 ms) : 1732, 1777
.   : milestone, 1755,
iast (1.509 ms) : 1485, 1534
.   : milestone, 1509,
profiling (1.55 ms) : 1525, 1576
.   : milestone, 1550,
tracing (1.484 ms) : 1459, 1509
.   : milestone, 1484,
section candidate
no_agent (1.357 ms) : 1338, 1376
.   : milestone, 1357,
appsec (1.746 ms) : 1722, 1770
.   : milestone, 1746,
appsec_no_iast (1.737 ms) : 1712, 1762
.   : milestone, 1737,
iast (1.505 ms) : 1481, 1529
.   : milestone, 1505,
profiling (1.513 ms) : 1490, 1536
.   : milestone, 1513,
tracing (1.496 ms) : 1472, 1520
.   : milestone, 1496,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.369 ms [1.349 ms, 1.389 ms] -
appsec 1.748 ms [1.724 ms, 1.772 ms] 379.351 µs (27.7%)
appsec_no_iast 1.755 ms [1.732 ms, 1.777 ms] 385.996 µs (28.2%)
iast 1.509 ms [1.485 ms, 1.534 ms] 140.587 µs (10.3%)
profiling 1.55 ms [1.525 ms, 1.576 ms] 181.611 µs (13.3%)
tracing 1.484 ms [1.459 ms, 1.509 ms] 115.572 µs (8.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.357 ms [1.338 ms, 1.376 ms] -
appsec 1.746 ms [1.722 ms, 1.77 ms] 388.867 µs (28.7%)
appsec_no_iast 1.737 ms [1.712 ms, 1.762 ms] 379.842 µs (28.0%)
iast 1.505 ms [1.481 ms, 1.529 ms] 148.073 µs (10.9%)
profiling 1.513 ms [1.49 ms, 1.536 ms] 155.778 µs (11.5%)
tracing 1.496 ms [1.472 ms, 1.52 ms] 139.049 µs (10.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/check-debugger
git_commit_date 1737139405 1737358992
git_commit_sha 0b239ae 8718354
release_version 1.46.0-SNAPSHOT~0b239ae5cc 1.46.0-SNAPSHOT~8718354560
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1737360944 1737360944
ci_job_id 772237336 772237336
ci_pipeline_id 53446944 53446944
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~8718354560, baseline=1.46.0-SNAPSHOT~0b239ae5cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (2.381 ms) : 2338, 2425
.   : milestone, 2381,
iast (2.111 ms) : 2056, 2166
.   : milestone, 2111,
iast_GLOBAL (2.161 ms) : 2106, 2217
.   : milestone, 2161,
profiling (1.966 ms) : 1922, 2009
.   : milestone, 1966,
tracing (1.951 ms) : 1909, 1993
.   : milestone, 1951,
section candidate
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.38 ms) : 2337, 2424
.   : milestone, 2380,
iast (2.117 ms) : 2062, 2172
.   : milestone, 2117,
iast_GLOBAL (2.163 ms) : 2108, 2218
.   : milestone, 2163,
profiling (1.993 ms) : 1948, 2038
.   : milestone, 1993,
tracing (1.958 ms) : 1916, 2001
.   : milestone, 1958,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.464 ms, 1.487 ms] -
appsec 2.381 ms [2.338 ms, 2.425 ms] 906.172 µs (61.4%)
iast 2.111 ms [2.056 ms, 2.166 ms] 635.822 µs (43.1%)
iast_GLOBAL 2.161 ms [2.106 ms, 2.217 ms] 685.967 µs (46.5%)
profiling 1.966 ms [1.922 ms, 2.009 ms] 490.588 µs (33.3%)
tracing 1.951 ms [1.909 ms, 1.993 ms] 475.564 µs (32.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.464 ms, 1.487 ms] -
appsec 2.38 ms [2.337 ms, 2.424 ms] 904.84 µs (61.3%)
iast 2.117 ms [2.062 ms, 2.172 ms] 641.315 µs (43.5%)
iast_GLOBAL 2.163 ms [2.108 ms, 2.218 ms] 687.363 µs (46.6%)
profiling 1.993 ms [1.948 ms, 2.038 ms] 517.352 µs (35.1%)
tracing 1.958 ms [1.916 ms, 2.001 ms] 482.839 µs (32.7%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~8718354560, baseline=1.46.0-SNAPSHOT~0b239ae5cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.945 s) : 14945000, 14945000
.   : milestone, 14945000,
appsec (14.991 s) : 14991000, 14991000
.   : milestone, 14991000,
iast (18.832 s) : 18832000, 18832000
.   : milestone, 18832000,
iast_GLOBAL (18.122 s) : 18122000, 18122000
.   : milestone, 18122000,
profiling (15.286 s) : 15286000, 15286000
.   : milestone, 15286000,
tracing (14.606 s) : 14606000, 14606000
.   : milestone, 14606000,
section candidate
no_agent (14.911 s) : 14911000, 14911000
.   : milestone, 14911000,
appsec (15.047 s) : 15047000, 15047000
.   : milestone, 15047000,
iast (18.668 s) : 18668000, 18668000
.   : milestone, 18668000,
iast_GLOBAL (17.944 s) : 17944000, 17944000
.   : milestone, 17944000,
profiling (15.196 s) : 15196000, 15196000
.   : milestone, 15196000,
tracing (14.962 s) : 14962000, 14962000
.   : milestone, 14962000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.945 s [14.945 s, 14.945 s] -
appsec 14.991 s [14.991 s, 14.991 s] 46.0 ms (0.3%)
iast 18.832 s [18.832 s, 18.832 s] 3.887 s (26.0%)
iast_GLOBAL 18.122 s [18.122 s, 18.122 s] 3.177 s (21.3%)
profiling 15.286 s [15.286 s, 15.286 s] 341.0 ms (2.3%)
tracing 14.606 s [14.606 s, 14.606 s] -339.0 ms (-2.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.911 s [14.911 s, 14.911 s] -
appsec 15.047 s [15.047 s, 15.047 s] 136.0 ms (0.9%)
iast 18.668 s [18.668 s, 18.668 s] 3.757 s (25.2%)
iast_GLOBAL 17.944 s [17.944 s, 17.944 s] 3.033 s (20.3%)
profiling 15.196 s [15.196 s, 15.196 s] 285.0 ms (1.9%)
tracing 14.962 s [14.962 s, 14.962 s] 51.0 ms (0.3%)

@smola smola marked this pull request as ready for review January 17, 2025 15:16
@smola smola requested review from a team as code owners January 17, 2025 15:16
@smola smola requested review from evanchooly and ygree and removed request for a team January 17, 2025 15:16
@smola smola force-pushed the smola/check-debugger branch from 2b19a80 to f51acda Compare January 20, 2025 07:02
@smola smola force-pushed the smola/check-debugger branch from f51acda to 8718354 Compare January 20, 2025 07:43
@smola smola enabled auto-merge (squash) January 20, 2025 08:17
@smola smola added comp: testing Testing and removed comp: testing Testing labels Jan 20, 2025
@smola smola merged commit a69247f into master Jan 20, 2025
178 of 179 checks passed
@smola smola deleted the smola/check-debugger branch January 20, 2025 11:49
@github-actions github-actions bot added this to the 1.46.0 milestone Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jpbempel jpbempel jpbempel approved these changes

@evanchooly evanchooly Awaiting requested review from evanchooly evanchooly is a code owner automatically assigned from DataDog/debugger-java

@ygree ygree Awaiting requested review from ygree ygree was automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

comp: testing Testing tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.46.0

Development

Successfully merging this pull request may close these issues.

3 participants

@smola @jpbempel